Open Science Essentials (1)

Ein Überblick zu offener Forschung mit ausgewählten Hands-on Vertiefungen

Jürgen Schneider

17 January 2023

If you like the workshop…

and want to keep it forever, make it yours
give it a star in GitHub

For that…

Fork the github repo this Quarto book is based on
Go to settings of your new repo and go to the “pages” section. Then set the “Branch” option to gh-pages (leave the dropdown to the right of this at /root)
Wait a minute to let the website get deployed. You can check on the status in the “Actions” tab of your repo.
Back on the main repo site, click on “About” (top right). In the URL of the website, change “j-5chneider” to your username “[your github username].github.io/OS-PHKA/” (you might need to activate GitHub Pages for that, by creating a GitHub Pages repo)
open your new webpage by clicking on that link in the “About” section

So you get noticed if I update something on the github repo.

And I get that sweet sweet dopamine.

Transparency

DIPF | Leibniz Institute for Research and Information in Education
Project in the Cooperation Center ShaReD - Sharing and Reusing Data

See personal webpage

Cooperation
Working closely together, but not at the research data center at DIPF

Research paradigm
Very much focused on quantitative-empirical research

Normalizing disbility (I guess)
I didn’t fall asleep, I am one the 1% who stutter (sometimes) ¯_(ツ)_/¯

Introduction and overview

current

challenges

new

technological

advancements

Example of a challenge

https://doi.org/10.1016/j.compedu.2023.104986

https://doi.org/10.1177/09567976231198184

https://doi.org/10.1007/s10648-023-09829-y

https://doi.org/10.1002/sce.21825

Example of a challenge

(Boudry et al., 2019)

Example of a technological advancement

Imagine: You analyze your data and…

someone wants to use the results for a meta-analysis
a reviewer wants to check your analyses
a team member wants to expand on your analyses
…

In any case: You’d need to share your analyses (and data).
How would you do it?

One option: Provide a browser-based instance of your R-project.
View example.

Why is Open Science such a thing right now?

current

challenges

new

technological

advancements

good

scientific

practice

What is Open Science?

an umbrella term

Open Definition

“Open means anyone can freely access, use, modify, and share for any purpose (subject, at most, to requirements that preserve provenance and openness).”
(Open Knowledge Foundation, 2023)

term grew and encompasses all disciplines (not just sciences) (UNESCO, 2022)
probably more accurate term: open scholarship

Open Science as a broad concept: 5 lines of reasoning

https://prezi.com/view/a3Jfa8g6X6bWxbaWXRoS/

Pragmatic School: Quality & development of research

Challenges & technological advancements

Replicability

Reuse

Transparency / intersubjective comprehensibility

Pragmatic School: Quality & development of research

Challenges & technological advancements

Replicability

same methods, new data = similar results (Barba, 2018)
“replication crisis”: disappointing results with several large-scale replications (Camerer et al., 2018; Errington, Mathur, et al., 2021; Open Science Collaboration, 2015)
Higher replication rates possible! (Protzko et al., 2023) if,
- Open data
- Open (reproducible) code
- Open materials
- Preregistration
Many of these are not the case in typical research (Artner et al., 2021; Errington, Denis, et al., 2021)

Reuse

Transparency / intersubjective comprehensibility

Pragmatic School: Quality & development of research

Challenges & technological advancements

Replicability

Reuse

Possibilities

Secondary analyses: fully exhaust analysis potential
saves resources
Meta-analyses
makes estimations easier
Teaching
enables practice-oriented education

Transparency / intersubjective comprehensibility

Pragmatic School: Quality & development of research

Challenges & technological advancements

Replicability

Reuse

Transparency / intersubjective comprehensibility

Transparency: Decisions in the research process are fully disclosed (Elliott & Resnik, 2019)

Enables independent researchers to evaluate scientific claims (Errington, Denis, et al., 2021)
Protects us from “questionable research practices” (Fiedler & Schwarz, 2016)
can increase trust in research/researcher (Schneider et al., 2022; Song et al., 2022)

References

Artner, R., Verliefde, T., Steegen, S., Gomes, S., Traets, F., Tuerlinckx, F., & Vanpaemel, W. (2021). The reproducibility of statistical results in psychological research: An investigation using unpublished raw data. Psychological Methods, 26(5), 527–546. https://doi.org/10.1037/met0000365

Barba, L. A. (2018). Terminologies for Reproducible Research (arXiv:1802.03311). arXiv. https://arxiv.org/abs/1802.03311

Boudry, C., Alvarez-Muñoz, P., Arencibia-Jorge, R., Ayena, D., Brouwer, N. J., Chaudhuri, Z., Chawner, B., Epee, E., Erraïs, K., Fotouhi, A., Gharaibeh, A. M., Hassanein, D. H., Herwig-Carl, M. C., Howard, K., Kaimbo Wa Kaimbo, D., Laughrea, P.-A., Lopez, F. A., Machin-Mastromatteo, J. D., Malerbi, F. K., … Mouriaux, F. (2019). Worldwide inequality in access to full text scientific articles: The example of ophthalmology. PeerJ, 7, e7850. https://doi.org/10.7717/peerj.7850

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikensten, E., Hummer, L., Imai, T., … Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644. https://doi.org/10.1038/s41562-018-0399-z

Elliott, K. C., & Resnik, D. B. (2019). Making Open Science Work for Science and Society. Environmental Health Perspectives, 127(7), 075002. https://doi.org/10.1289/EHP4808

Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Challenges for assessing replicability in preclinical cancer biology. eLife, 10, e67995. https://doi.org/10.7554/eLife.67995

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. eLife, 10, e71601. https://doi.org/10.7554/eLife.71601

Fiedler, K., & Schwarz, N. (2016). Questionable Research Practices Revisited. Social Psychological and Personality Science, 7(1), 45–52. https://doi.org/10.1177/1948550615612150

Open Knowledge Foundation. (2023). Open Definition 2.1. https://opendefinition.org/od/2.1/en/.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716–aac4716. https://doi.org/10.1126/science.aac4716

Protzko, J., Krosnick, J., Nelson, L., Nosek, B. A., Axt, J., Berent, M., Buttrick, N., DeBell, M., Ebersole, C. R., Lundmark, S., MacInnis, B., O’Donnell, M., Perfecto, H., Pustejovsky, J. E., Roeder, S. S., Walleczek, J., & Schooler, J. W. (2023). High replicability of newly discovered social-behavioural findings is achievable. Nature Human Behaviour. https://doi.org/10.1038/s41562-023-01749-9

Schneider, J., Rosman, T., Kelava, A., & Merk, S. (2022). Do Open-Science Badges Increase Trust in Scientists Among Undergraduates, Scientists, and the Public? Psychological Science, 33(9), 1588–1604. https://doi.org/10.1177/09567976221097499

Song, H., Markowitz, D. M., & Taylor, S. H. (2022). Trusting on the shoulders of open giants? Open science increases trust in science for the public and academics. Journal of Communication, 72(4), 497–510. https://doi.org/10.1093/joc/jqac017

UNESCO. (2022). UNESCO Recommendation on Open Science.

Credit

Title: Adrian Infernus on Unsplash

Stuttering Pride Flag by Conor Foran licensed under CC BY-NC-SA 4.0

Icons: CC BY 4.0 Fontwaesome